Artificial neural networks (ANNs) are computing systems inspired by the biological neural networks that constitute animal brains. Such systems learn and progressively improve performance on tasks by considering and training from examples.
An ANN is based on a collection of connected units or nodes called artificial neurons. Each connection (synapse) between neurons can transmit a signal from one to another. The receiving (postsynaptic) neuron can process the signal(s) and then signal neurons connected to it.
In common ANN implementations, the synapse signal is a real number, and the output of each neuron is calculated by a non-linear function of the sum of its inputs. Neurons and synapses typically have a weight that adjusts as learning proceeds. The weight increases or decreases the strength of the signal that it sends across the synapse. Neurons may have a threshold such that the signal is only sent if the aggregate signal crosses the threshold.
Typically, neurons are organized in layers. Different layers may perform different kinds of transformations on their inputs. Signals travel from the first (input) to the last (output) layer, possibly after traversing the layers multiple times.
In machine learning, a convolutional neural network (ConvNet) is a class of deep, feed-forward artificial neural networks that has successfully been applied to analyzing visual imagery.
ConvNet use a variation of multilayer perceptrons designed to require minimal pre-processing. They are also known as shift invariant or space invariant artificial neural networks (MANN) based on their shared-weights architecture and translation invariance characteristics.
ConvNet use relatively little pre-processing compared to other image classification algorithms. This means that the network learns the filters that in traditional algorithms were hand-engineered.
Neural networks typically use a matrix vector linear product between layers with a common function applied element wise to the resulting vector function which may result in non-linear layers such as radial basis or threshold rectified layers. However, more complex non-linear layers requiring a per element function such as Fourier decompositions, polynomial or absolute value or multiple thresholds are difficult to implement and the back propagation through such layers is not always solvable.
In addition, the result of layers replaces the input data and is not an addition to the input data. Multiple paths in neural networks have to be added if the previous input needs to be propagated such as with Residual Neural Networks.